To View The Abstract, Please Click On PDF File.

Many hierarchical techniques to solve large MARKOV decision PROCESSES (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorithm, which is a variant of Tarjan’s algorithm that simultaneously finds the SCCs and their belonging levels. Second, a new definition of the restricted MDPs is presented to ameliorate some hierarchical solutions in discounted MDPs using value iteration (VI) algorithm based on a list of state-action successors. Finally, a robotic motion-planning example and the experiment results are presented to illustrate the benefit of the proposed decomposition algorithms.

Production and non-productive equipment and personnel delays are a critical element of any production system. The frequency and length of delays impact heavily on the production and economic efficiency of these systems. Machining PROCESSES in wood industry are particularly vulnerable to productive and non-productive delays. Whereas, traditional manufacturing industries usually operate on homogeneous raw material, in a restricted environment with closely controlled processing guidelines. The logging industry must continually deal with a raw material that comes in many different shapes, sizes and performs in an environment that is different from site to site. Furthermore, loggers; rarely have the opportunity to follow a predetermined production sequence, as men and machines must maneuver as conditions dictate. As a result machining systems can experience a broad range of delays that vary widely in frequency and length. The purpose of this study was to apply MARKOV process models to the analysis of delay times in machining PROCESSES. Such an approach will permit the random components of machining process to be integrated into a flexible mathematical model, using theoretical probability distributions, and providing analytic solutions to proportions of productive delay time.

In the present paper we investigate the L1-weak ergodicity of nonhomogeneous continuous-time MARKOV PROCESSES with general state spaces. We provide a necessary and sufficient condition for such PROCESSES to satisfy the L1-weak ergodicity. Moreover, we apply the obtained results to establish L1-weak ergodicity of quadratic stochastic PROCESSES.

Stochastic differential equations (SDEs), arise from physical systems that possess inherent noise and certainty. We derive a SDE for electrical circuits. In this paper, we will explore the close relationship between the SDE and autoregressive (AR) model. We will solve SDE related to RC circuit with using of AR (1) model (MARKOV process) and however with Euler-Maruyama (EM) method. Then, we will compare this solutions. Numerical simulations in MATLAB are obtained.

In this paper, the two parameter ADK entropy, as a generalized of Re’nyi entropy, is considered and some properties of it, are investigated. We will see that the ADK entropy for continuous random variables is invariant under a location and is not invariant under a scale transformation of the random variable. Furthermore, the joint ADK entropy, conditional ADK entropy, and chain rule of this entropy is discussed. The ADK entropy rate is defined and is used for deriving the entropy rate of stationary Gaussian PROCESSES and an irreducible- aperiodic MARKOV chain.

This paper introduces MARKOV-switching (MS) GARCH PROCESSES for capturing the skewness in the distribution of financial time series. The model class is motivated by the fact that empirical return distributions characterized by significant asymmetries, but the generic assumption of return distributions is Normal. The out of sample performance of symmetric and asymmetric MS GARCH models is compared in an application to Tehran exchange price index (Tepix). Finally, to put the Regime-switching models into perspective, we add to the list of competitors a popular model which may serve as a benchmark, i.e., the single–regime GARCH (1, 1). It turns out that asymmetric MS GARCH PROCESSES perform best overall.